The enhancement of sensor precision is vital for improving the control performance of servo system. The measurement error of resolver is studied. The measurement error of internal angle tracking loop of resolver is analyzed and it is found that major part of error is even harmonics. The precision of the resolver is compensated with polynomial fitting, Fourier decomposition and table look-up method. The practical product test results show that the precision of resolver is sharply improved.